Controlled Abstention Neural Networks for Identifying Skillful Predictions for Regression Problems

The earth system is exceedingly complex and often chaotic in nature, making prediction incredibly challenging: we cannot expect to make perfect predictions all of the time. Instead, look for specific states that lead more predictable behavior than others, termed “forecasts opportunity.” When these opportunities are not present, scientists need systems capable saying “I don't know.” We introduce a novel loss function, “abstention loss,” allows neural networks identify forecasts opportunity regression problems. abstention works by incorporating uncertainty network's confident samples abstain (say know”) on less samples. designed determine optimal fraction, or user-defined fraction using standard adaptive controller. Unlike many methods attaching network post-training, applied during training preferentially learn from built upon nonlinear heteroscedastic regression, computer science method. While simple yet powerful tool problems, demonstrate outperforms it synthetic climate use cases explored here. implementation proposed straightforward most architectures as only requires modification output layer function.